Multimedia computer software products present sounds, graphic images, animation and movies to their users in an interactive environment. Typically, users can explore and revisit different sections of the multimedia product. Usually, a multimedia product is designed so that as a user advances through the content of the product, the user navigates through information, solves puzzles, demonstrates/tests knowledge, or participates in interactive simulations.
The development of a multimedia product is typically achieved through the joint efforts of skilled artists who produce the sounds, images and animations, and skilled software authors who bind together the art, define the interactive relationships between the multimedia product and the user, and codify the behavior of the simulations and games. A software author's job is typically complex, generally entailing a relatively steep learning curve.
Software authors typically use an authoring tool to create or modify a multimedia product. Rather than requiring an exact text-based syntax as is found in a programming environment such as Apple's MPW or Symantec's Think C, authoring tools generally provide a direct-manipulation interface that allows authors to define the behavior of objects by writing a textual description in a simple scripting language, or by rearranging graphic items in a flowchart, screen map, score, or other representation. Examples of authoring tools include HyperCard, AuthorWare Professional, Macromedia Director, Visual Basic or Apple Media Tool. HyperCard and Apple Media Tool are available from Apple Computer, Inc. HyperCard and Apple are registered trademarks of Apple Computer, Inc. AuthorWare Professional and Director are available from Macromedia and Visual Basic is available from Microsoft Corporation. Authorware Professional and MacroMedia Director are registered trademarks of Macromedia, Inc. and Visual Basic is a registered trademark of Microsoft Corporation.
In an aspect of authoring tools, software titles and content created using the tools are comprised of one or more stages where at any given time a single stage of the software title, is presented, e.g. displayed, to the end-user. Typically, as a user advances through a multimedia title, different stages are consecutively presented depending on the actions of the end-user as he/she progresses through the software title.
For example, a multimedia product might present the user with a picture containing one or more elements. As the user interacts with the elements in the picture, the elements change. For example, the picture may present a scene in which there is a dog and a cat, each in a standing position. If the user positions the cursor over the dog and depresses a mouse button, the dog smiles. If the user, repeats that action, the dog sits. The scene with the dog smiling and the scene with the dog sitting are each different states of the multimedia product.
Each element may have a unique set of behaviors associated with it. For example, a multimedia product might display a room with a dresser with a plurality of drawers, each drawer containing multiple interactive elements, e.g. games, boxes. To access the interactive elements inside the dresser's drawers, a user opens a drawer on the dresser, thereby exposing the interactive elements contained therein. The user can then interact with those elements, along with the other elements being displayed.
Usually an authoring tool is presented via a graphic metaphor. For example, in HyperCard the graphic metaphor is a stack of cards, while in Authorware Professional a flow chart metaphor is used. In MacroMedia Director a movie score graphic metaphor is used. In the above metaphors, it is typically cumbersome to implement a single screen with multiple interactive graphic elements. For example, in a stack metaphor, a separate card is typically created for each possible permutation of the display of the picture. Similarly, in a flow chart metaphor a separate graphic image is usually created for each permutation.
Typically, the number of possible permutations is a function of the number of elements and the number of possible positions for each element. Where each element has the same number of possible positions, the number of permutations is equal to N.sup.m, where N equals the number of different positions for each object and m equals the number of elements. As the number of interactive elements and/or the number of possible positions increase, so too does the number of permutations.
For example, in HyperCard to implement a picture having interactive elements such as the cat and dog previously described, a card is typically created for each possible permutation. For example, if the cat and dog elements start out standing, but in response to user interaction can be individually made to smile and sit, then a card is produced for each possible permutations. In the above cat and dog example there are 9 possible permutations: the cat and dog standing, the cat standing and the dog smiling, and vice versa, the cat and dog smiling, the cat sitting and the dog smiling, and vice versa, the cat sitting and the dog standing, and vice versa, and the cat and dog sitting. Alternatively, the above can be implemented using the HyperTalk scripting language with XCMD extensions. For detailed information on HyperCard refer to "The Complete HyperCard 2.2 Handbook", fourth edition, Danny Goodman, Random House, 1993.
In MacroMedia Director, the situation in which interactive elements have behavior associated with it, an author of a multimedia title typically uses the LINGO scripting language to script the behavior of the interactive elements.
Thus, it is desirable to have a mechanism whereby authors can develop multimedia titles having interactive elements without using a scripting language and without having to implement a screen for each permutation of the multimedia title.
Some authoring tools such as AuthorWare Professional apply a flowchart model. However, these tools typically apply a linear flow and do not provide an easy mechanism for jumping to an arbitrary part of the multimedia product.
For example, in the situation where a stand-alone computer, called a "kiosk", is used to provide information on a particular topic, e.g. a museum, shopping center, store, it may be desirable to design the multimedia title to return to the beginning in certain circumstances, e.g. after a predetermined amount of inactivity has lapsed or a new user has approached the kiosk. In an authoring tool which applies a linear flow, this can be difficult to implement, especially if there are a substantial number of different screens in the multimedia title.
In some authoring tools such as the Apple Media Tool, a linear order is not imposed. In the Apple Media Tool, as shown in FIG. 1, a display area shows a diagram illustrating the relationship between the screens of a multimedia title. Each screen is represented by a screen icon in the shape of a rectangle and the operative relationship between the screens is denoted by a line with one or more arrows indicating the relationship's flow. For example, the line between screen icon 4 and screen icon 2 indicates that in operation, screen 2 may be presented immediately after screen 4. A screen can have a plurality of incoming and outgoing relationships with other screens. Screen 2 has an outgoing relationship with screens 1, 3 and 4. The screen which follows screen 2 typically depends on the user's interaction with screen 2. In some cases, screen 1 immediately follows screen 2, while in other cases screen 3 or screen 4 follows.
The Apple Media Tool further includes a tool palette, as shown on the left of FIG. 1. The tool palette contains tools for creating screen icons, creating and deleting relationships between screens. A user creates a screen icon by selecting the appropriate tool and then specifies a location within the display area at which to display the new screen icon. A user creates a relationship between two screens by clicking on the appropriate tool and then specifying the two screens that are part of the relationship.
Although tools such as the Apple Media Tool allow for screens to be arbitrarily arranged, when a title is running the associated graphics are the same size for each screen, typically each graphic filling an entire display area. Thus, a screen having a plurality of interactive elements, such as the dog and cat example previously described, can not generally be created using the iconic user interface shown in FIG. 1. However, the functionality can be achieved using scripting tools.
Some tools such as Camera Man from Motion Works record the development user interaction with a computer system, but they typically record the interaction at a detailed level, such as the entire screen of each interaction. Moreover, the recording is not tied to particular objects or actions, so that it is difficult to designate replay of a subset of the recorded material where the subset is tied to a particular action, object or other specification.
In another aspect of authoring tools, authors use the tools to manipulate various types of objects and events when creating or modifying a multimedia product. For example, authors manipulate text, graphic objects such as buttons or pictures, and transient events such as sounds, video, visual effects, animations, interapplication communications and process invocations. Unlike graphic objects which are easy to manipulate graphically during authoring because they have a visual representation at runtime, transient events are not graphic or tangible at runtime and, thus, are difficult to locate and work with while authoring.
For example, consider the situation in which a graphic drawing is being displayed in a window. In particular, on a computer running an Apple's System 7.5 operating system, a jigsaw puzzle window containing a graphic drawing can be displayed. To replace the graphic drawing, displayed in the jigsaw puzzle with another graphic drawing, a user can drag a graphic object from the Scrapbook onto the jigsaw puzzle window and drop it in the window, thereby causing the new graphic image to be displayed in the jigsaw puzzle window in place of the old graphic image. In this manner a graphic drawing of a dog in the jigsaw puzzle window can be replaced by a graphic image of a cat from the scrapbook by directly manipulating the graphic drawings.
Unlike pictures, sounds cannot be directly manipulated graphically, because they do not have a direct graphic representation. Authors can manipulate a reference to a sound, e.g. a text description of the sound, but can not directly manipulate the sound itself. For example, the text "meow" can be used as a reference to a cat sound and the text "bark" can be used as a reference to dog sound. An author can replace a bark with a meow by manipulating the text reference "meow". However, the use of a representation that is different from the true audio-nature of the sound presents several problems. First, the text may be language-specific where the sound is not. For example, the sound of a cat may be textually represented differently depending on the language. In English "meow" is used, while in Portuguese "miau" is used. Second, the text description may be inaccurate. In manipulating a text description, an author does not experience the transient event, e.g. the sound playing, so that the author could erroneously use the wrong sound, e.g. a chiwawa barking rather than a bark of a german shephard.
Authoring tools typically take one of two approaches. Some authoring tools represent a sound using a text string and allow authors to choose sounds by selecting them from a presented list of text strings. For example, a beep sound may be represented by the text string "beep" and a whistle sound may be represented by the text string "whistle". Since the sounds are not represented in the context of their use and are not in their actual representations, it can be difficult for an author to identify which sound to use. Moreover, it is sometimes difficult for an author to conceive of a name that appropriately and/or effectively describes a particular sound.
Other authoring tools assume that an author can remember where in the multimedia product a transient event is evoked. The author then traces through the code to try to determine how to reference the particular transient event. However, in situations where the code may be complex and/or is comprised of many interdependent layers, the task of tracing through code to find a particular transient event can be quite difficult. Moreover, the difficulty of the task can be further amplified if an author does not remember where in the multimedia product a transient event is evoked, thereby increasing the field of the search.
Both approaches for handling transient events are difficult for authors and for novice end-users who simply want to tweak the multimedia product.